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(54) Automatic image data quality adjustment to reduce response time of a Web server 



(57) The present invention provides methods, de- 
vices and systems for dynamically adjusting transcod- 
ing parameters so as to increase the benefits of trans- 
coding. Methods of adaptation are designed to cope 
with the variability of network characteristics and of the 
size of transcoded images. The invention also provides 
a method and apparatus to enable the transcoding 
proxy to adjust a quality-size tradeoff on a per-image 



and/or a per-client basis. The adaptive transcoder 
chooses different parameters for each object, and pro- 
vides performance improvements. The invention further 
provides a general framework for making policy deci- 
sions taking into account available bandwidth, content 
and type of image, and user preferences. The invention 
also includes methods for generating feedback about 
the choice of optimal transcoding parameters to the us- 
er. 
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[0 001] This^ventionre.ates^^ 

generally related to any proxy or gateway system that P«^^J2b2Si1Sl variety of client devices 
?0002] Transcoding proxies sometimes act 35 ' nte ™ d ^^^ transcoding proxy pro- 

the client device. . Ko„H«,irtth «/imiP«!«5 or medium bandwidth wireline modem 

[0003] Mobile devices are frequently connected via tow-bandwidth wire ^ "^'r 1 ™ Q long d0wnlO ad 
inks that make the viewing of rteh web content, such as ""J*?*^^ networks, 
times that result. In addition, the cost of such downloads can be 'P'«e ^^J^^ Download time 
Transcoding web proxies can reduce the ^^^ZXZg^^^ ***** 
reductions of six to ten times may be achieved for many W«" *f D » some times also capable of tailoring text 
[0004] The second important benefit is that such an intermediate .proxy bs that are now available. The 

Lnd images forthe multitude of small, weakly connected,bu^ 

capabilities of these mobile devices to recerve, P™)^ devices. Consequent*, 

of client devices, it is dCTicutt for Internet content P u ^ ,shers h t0 * ^ ™ resolution color-depth, and size 

an active web proxy is used to transcode/change ^^^^^^^ its original size, 
constraints of a small-screened device and to reduce the ^^^f^^^^, in an HT TP proxy. The 
[0005] Atypk^tianscodingwebprax^ 

LJoding module is an a-rdlng to a set of 

^^^^^^^-^^ *■ reso,ution toss and 

2TL P of the object. The fo.lowing discussion ^^^^S^ si2e tradeoff such that the 
[0006] To maximize the benefits of transcodmg, ,t ,s important^ to_ c hoose tte^ualrty -V transcoding pa . 

best quality image is sent within the delay tolerance 'f^.^^^^^ s ? ze because the degree of 
rameters, however, is difficuK. First, it is hard to «« can be compressed 

coding may be wasteful. ^.^-i notwnrk variabilitv makes it hard to predict image 

[0007] Even if the size of the transcoded output can be predicted. "^^^ ™™ sureSi accuiate estimates 

adjustments. . ^ m „ir>atori ta*k is the location of the bottleneck in the network. 

[0008] Another issue that makes policy decisis a comply 

due to transcoding. ^ ra ^ri=ti^c and imaae content transcoding proxies may 

[0009] Due to the difficulty of estimating the network characteristics and image c °" len ^ paramete re which are 
often only support adaptation at a very coarse level. Adapter, involves ° h ^j^^^ is tn at 

usually selected from a predefined set of defaults, or c ^ e " network band- 

^^^^^^^^^^^^ 

process is time consuming and not very user-fnendly. . . k nd var j e d client devices. As 

[00,0, Fig.. 1 show* an .xampl* ™ ' 30 '' 34 

Lwn « Ffcu,. . , «» ~n S M„ an ™ ™ so.n«rio ol Figure 1 . « 
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data over low-speed wide-area wireless links, like Cellular Digital Packet Data (CDPD) 163 that provides 10 kb/s or 
less of throughput, depending upon the number of clients sharing the link. The end client could be a full-function PC 
1 34 with a good color display, so that the primary problem is bandwidth reduction. Alternatively, the client might be a 
small, web-enabled, mobile devices 130-132 in which case it is advantageous to tailor the web data for the specific 
5 client device, particularly the client's display characteristics. Different aspects of transcoding proxy design benefit dif- 
ferent scenarios (i.e., tariffed proxy-client link, strong/weak client display, etc.), but all are handled well within the same 
proxy architecture 1 00. 

[001 2] Figure 2 shows a block diagram of an embodiment of transcoding proxy 1 90 used to transform objects based 
on user specified preferences and static policies. A transcoding proxy 1 90 is built by combining a transcoding module 

10 240 with an HTTP proxy engine 220. An HTTP request 222 originates from a client 230 and is forwarded 224 by the 
proxy 1 90 to a web server 21 0. The response data 226 (i.e., HTML pages and GIF and JPEG images) are transformed 
by the transcoder 240 and then forwarded 228 to the client 230. Typically, a number of transcoding parameters are 
specified to the transcoder 240 in order to achieve the desired quality/size reduction of the object contained in the 
response data 226. Transcoding proxies in use today either use a static set of policies 250 or use some form of user 

is specified preferences 260 via path 265 to determine the transcoding parameters. When a fixed set of transcoding 
parameters are applied to all objects, results are not always beneficial. In fact, in many cases, transcoding leads to 
poorer performance. 

[0013] Accordingly, in a first aspect, the present invention provides a method for a transcoding proxy to facilitate 
browsing between a plurality of client devices and a plurality of servers connected via a communication network, the 

20 method comprising: receiving a request from one of the client devices for an object stored at one of the servers, for- 
warding the request for the object to said one of the servers, receiving the object from said one of the servers, examin ing 
preferences specified by a user of said one of the client devices, examining contents of the object, examining commu- 
nication network characteristics, choosing a set of transcoding parameters, forming a transcoded form of the object, 
and sending the transcoded form to said one of the clients. 

25 [0014] Preferably, the network characteristics include bandwidth, and examining network characteristics includes 
estimating network bandwidth between said one of the servers and the proxy as well as between the proxy and .said 
one of the clients. 

[0015] Preferably, the network characteristics include delay, and the step of examining network characteristics in- 
cludes estimating delay between said one of the servers and the proxy as well as delay between the proxy and said 
30 one of the clients. 

[0016] The method preferably further comprises providing feedback to the user about a level of transcoding per- 
formed on the object to form the transcoded form. 

[0017] Preferably, the step of examining includes determining the size of the object. 

[0018] In the method as described the object may be of type image forming an image object, and the method further 
35 comprises: determining dimensions of the image object, and calculating the compression ratio of the image object. 
Further to be preferred is that the dimensions of the image object are determined by area of the image in square pixels, 
and the compression ratio is determined by the bpp ratio of the image object. 
[0019] The step of forming a transcoded form preferably employs dynamic adaptation. 

[0020] The step of forming a transcoded form is preferably started before the step of receiving the object from said 
40 one of the servers is complete. The received object type is preferably of type JPEG forming a JPEG object. Further to 
be preferred is that the step of forming a transcoded form includes performing JPEG4o-JPEG image transcoding and 
the step of sending the transcoded form starts writing out at least one MCU of JPEG-encoded output image data before 
the step of receiving the object is complete. 

[0021] In the method as described, the step of sending out the transcoded form is preferably started after processing 
45 an initial fraction of the received object, and before the step of receiving the image object from said one of the servers 
is complete. 

[0022] The step of sending out the transcoded form is preferably started before the step of forming a transcoded 
form of the object is complete. 

[0023] In a second aspect, the present invention provides a method for a proxy to form a transcoded form of an 
50 object received from a server in satisfaction of a request from a client for an object available from the server, the method 
comprising dynamically adapting parameters for transcoding the object for the client, forming a transcoded form of the 
object, and sending the transcoded form to the client. The step of adapting parameters preferably includes determining 
at least one characteristic of the object. One characteristic is preferably an object -header, the object-header providing 
information about the size and the type of the object. The method preferably further comprises comparing the size of 
55 the object to a threshold parameter called B size_threshold B . The step of adapting parameters preferably includes gath- 
ering present network characteristics between the server and the proxy and between the proxy and the client. One of 
the characteristics is preferably network bandwidth and the step of adapting includes estimating network bandwidth 
between the server and the proxy as well as between the proxy and the client. 
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100241 In the method as described, the transcoded form is preferably dependent upon the estimated bandwidth. The 
step of adapling also preferably includes retrieving preferences of the user, and wherein the transcoded tarn, us de- 
pendent upon the presences. The step of adapting preferably includes examining the contents of the object The 
objectis pSably of type image forming an image object, and examining the content of the image ob,ect includes 
determin in a dimensions of the image. * . . . 

[00251 In the method as described, the step of adapting is preferably dependent upon determining the compress^ 
ratio of the image object. The type of image object is further preferably GIF and wherein the step of adapt^g ,s de- 
pendent upon comparing the compression ratio against a predetermined policy threshold called 9 rf -^ esh °' d . 
[00261 The method as described preferably further comprises predicting at least one parameter of he transcoded 
form of the object. The transcoded form is further preferably the same as an original form of the object Preferably at 
least one of the parameters includes a size of the transcoded form. Preferably also, at least one of the parameters 
includes the time spent in transcoding the object. . . . fnm of 

r0O27] In a third aspect, the present invention provides a method for predicting parameters of a transcoded form of 
an ob ect, the object having an initial size and dimension, and the object being received from a server satisfaction 
of a request from a client for the object, the method comprising: computing the bpp ratio of the object gathering a set 
of statistics of a plurality of previously transcoded objects, and employing the set of statist.es and the bpp ratio for 

S'TaTZTdescribed, at least one of the parameters is preferably size and the set of statistics includes 
sizes of a plurality of previously transcoded objects statistics. . . . 

[0029] The object te preferably of type image and the set of statistics includes image quality. The plurality of previously 
transcoded objects are preferably chosen from a predetermined benchmark suite of images. efatictir< 
r0030] In a method as described, the step of employing preferably uses dynamically updating the set of statistics 
using the statistics of the currently transcoded object. At least one of the parameters is preferably a duration for forming 
a transcoded form of an object and the set of statistics includes the duration for forming a plurality of previously trans- 

r00 l 31 d | Ob | J n afourth aspect, the present invention provides a transcoding proxy system forfacilitating browsing between 
a plurality of clients and a plurality of servers connected via a communication network, the proxy comprising an HTTP 
proxy engine to receive a request from one of the clients for an object stored at one of the servers, and to fetch the 
object from said one of the servers, an object transcoder to form a transcoded form of the object using a set of param- 

30 eters for transcoding, a dynamic policy module to determine the set of parameters of transcoding, ar >>>™0e«" ^ 
delay predictor module to gather characteristics of the object, a user preference module to gather quality References 
specified by a user of said one of the clients, and a bandwidth estimation module to estimate available network band- 
width, wherein dynamic policy module dynamically adjusting the parameters of transcoding us.ng the mpu ^ceived 
from he image size and delay predictor module, user preferences module, and bandwidth 'estimation module for the 

35 purpose of improving satisfaction for the user, and the transcoding system providing feedback to the user about the 

level of transcoding performed. -,„ ,„~i...i~> 

r0032] The user preferences module preferably further collects the characteristics such as display size, resolution, 
& CPU speed of said one of the devices, and provides those characteristics to the dynamic policy module. 
r0033] In the system as described, the bandwidth estimation module preferably collects traces of previously estab- 

40 ished network connections between said one of the servers & the proxy, collects traces ol P"»^ ^"J* 
network connections between the proxy and said one of the clients, and estimates the object download time by per- 
forming statistical analysis on the collected traces. ^ thonm ™k nrB f 
r00341 The statistical analysis used for estimating bandwidth between said one of the servers and the proxy is pref- 
erably based on computing a statistical measure such as median, mean, or mode of download times of previously 

45 fetched objects as determined from the collected traces. 

[0035] The statistical analysis used for estimating bandwidth between the proxy and said one of the clients is pref- 
erably based on computing aggregate bandwidth of all active connections between the proxy and said one of the 
l^s ™%™e*> preferably further comprises displaying a slide bar on said one o. the client's display for collecting 
the user specified preferences. In a system as described, the user of said one of the clients can Preferably specie 

so tradeoff between download time and data quality through the use of a graphical user interface with a slide ban The 
user of said one of the clients can preferably specify through the use of a graphica. user interface With a s de bar he 
tradeoff between download time and image quality including a specific switch to select color or gray scale as tte <to red 
output form. The user of said one of the clients can further preferably specify through the use of a f^^ r 
interface with a slide bar, the desire to maintain a target response time such that the system automatically reduces 

ss data quality (and hence data download size) to compensate for dynamic variations in bandwidth to said on<= . o the 
clients. The graphical user interface slider bar is also preferably used as an output interface for showing the optimal 
choice of transcoding parameters to the user. 

[0036] In a fifth aspect, the present invention provides an article of manufacture comprising a computer usable me- 
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dium having computer readable program code means embodied therein for causing dynamic adaptation of transcoded 
form of an object in a transcoding proxy, the computer readable program code means in said article of manufacture 
comprising computer readable program code means for causing a computer to effect: a proxy receiving an object 
associated with a user from a server, determining parameters of the object, retrieving preferences of the user, gathering 
5 present network characteristics, obtaining transcoding policy thresholds, performing a policy decision based upon ob- 
ject parameters, user preferences, network characteristics, and policy thresholds, forming a transcoded object, pro- 
viding feedback of a level of transcoding performed on the object to the user, and sending the transcoded object to the 
user. 

[0037] In a sixth aspect the present invention provides an article of manufacture comprising a computer usable 
10 medium having computer readable program code means embodied therein for causing a transcoding proxy to facilitate 
browsing between a plurality of client devices and a plurality of servers connected via a communication network, the 
computer readable program code means in said article of manufacture comprising computer readable program code 
means for causing a computer to effect: receiving a request from one of the client devices for an object stored at one 
of the servers, forwarding the request for the object to said one of the servers, receiving the object from said one of 
is the servers, examining preferences specified by a user of said one of the client devices, examining contents of the 
object, examining communication network characteristics, choosing a set of transcoding parameters, forming a trans- 
coded form of the object, and sending the transcoded form to said one of the clients. 

[0038] The computer readable program code means in said article of manufacture preferably further comprises com- 
puter readable program code means for causing a computer to effect providing feedback to the user about a level of 

20 transcoding performed on the object to form the transcoded form. The computer readable program code means in said 
article of manufacture preferably further comprises computer readable program code means for causing a computer 
to effect determining dimensions of the object, and calculating the compression ratio of the object. The computer 
readable program code means in said article of manufacture preferably further comprises computer readable program 
code means for causing a computer to effect starting the step of forming a transcoded form before the step of receiving 

25 the object from said one of the servers is complete. 

[0039] In a seventh aspect, the present invention provides a computer program product comprising a computer 
usable medium having computer readable program code means embodied therein for causing a proxy to form a trans- 
coded form of an object received from a server in satisfaction of a request from a client for an object available from 
the server, the computer readable program code means in said computer program product comprising computer read- 

30 able program code means for causing a computer to effect: dynamically adapting parameters for transcoding the object 
for the client, forming a transcoded form of the object and sending the transcoded form to the client. 
[0040] The computer readable program code means in said computer program product preferably further comprises 
computer readable program code means for causing a computer to effect gathering present network characteristics 
between the server and the proxy and between the proxy and the client. The computer readable program code means 

35 jn said computer program product preferably further comprises computer readable program code means for causing 
a computer to effect adapting parameters for transcoding based upon the estimated bandwidth and preferences of the 
user. 

[0041] In an eighth aspect, the present invention provides a program storage device readable by machine, tangibly 
embodying a program of instructions executable by the machine to perform method steps for predicting parameters 
40 of a transcoded form of an object, the object having an initial size and dimension, and the object being received from 
a server in satisfaction of a request from a client for the object, said method steps comprising: computing the bpp ratio 
of the object, gathering a set of statistics of a plurality of previously transcoded objects, employing the set of statistics 
and the bpp ratio for predicting the parameters. 

[0042] Said method step of employing the set of statistics preferably further comprises updating the set of statistics 
is using the statistics of the currently transcoded objects. 

[0043] An aspect of the present invention is to provide methods, devices and systems for dynamically adjusting 
transcoding parameters so as to increase the benefits of transcoding. Methods of adaptation are designed to cope 
with the variability of network characteristics and of the size of transcoded images. 

[0044] In an embodiment, the invention includes three new components: an image size predictor, a network band- 
50 width (b/w) analyzer, and a policy module. Before initiating any transcoding action, the policy module queries the image 
size predictor to estimate the size of the output image. The b/w analyzer is queried to collect an estimate of the image 
transmission time from the server to the proxy, and from the proxy to the client. Based on the collected estimates, the 
proxy decides whether or not to transcode the image. In addition, the policy module can also compute the optimal point 
in the quality vs. size tradeoff that would provide the user specified performance criteria (e.g., reduced response time, 
55 increased quality). 

[0045] Another aspect of the present invention is to provide a method and/or apparatus to enable the transcoding 
proxy to adjust a quality-size tradeoff on a per-image and/or a per-client basis. The adaptive transcoder chooses dif- 
ferent parameters for each object, and provides performance improvements. 
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[0046] Sti„ another aspect of this invention provides a ^^^2^^ 
account available bandwidth, content and type of image, and user P<°<°™*" J h * * d J™JS , n one embodiment 
from a variety of optimization objectives so as to resources 

SS£SST*» Tee up users from active* controlling thj jpoj* , engine c, the = ^ 
[00471 in still another aspect of the present invent-on a t ^^^^^^ B system provides feedback 

SS? " prSrel^eTo^ invention w, now be described by way of examp.e, with reference to 
the drawings in which: 

FIG. 1 shows an example transcoding proxy scenario with varied links and varied client devices; 

F 1G .2showsa b lockdiagramofatrans^ 
and static policies; 

generator; 

F.G. 4 shows an example block diagram of an examp.e HTTP proxy with caching and transcoding modules in 
accordance with the present invention; 

PIG. 5 shows an examp.e f.ow diagram of a transcoding dynamic po.icy modu.e in accordance wfth the present 

invention; 

FIG. 6 shows an examp.e web request-response cycle using a transcoding proxy in accordance with the present 

30 invention; 

PIG. 7 shows an examp.e regime where transcoding is usefu. in accordance wrth the present invention; 
F.G. 8 shows an examp.e flow diagram of a examp.e policy function in accordance with the present invention; 
FIG. 9 shows an example b.ock diagram of an examp.e image size predion modu.e in accordance with the 
present invention; 

FIG. 10 shows an example b.ock diagram of an examp.e bandwidth prediction modu.e in accordance with the 
40 present invention; 

FIG. 11 shows input/feedback user interface in accordance with the present invention; and 
FIG. 12 il.ustrates an example timing digram of streaming transcoding in accordance w«h the present invents. 
[0049] Figure 3 shows a bkx* digram 300 of an —P^™ 

accordance wKh the present Mention. A ^mpanson of * £££ Z dynamte po.icy module 370 

module, 250 in Figure 2, with a & LsccWolicy 0-. the transcoding 

is to make decisions concerning when to turn tnnecodinG l°" a ™ «• interfaces with an image size and 

algorithm atong with its parameters) to use. The dynamc , n the embodiment shown, the 

delay predictor 375, a bandwidth estimator 380. and a ^\!^^Zts o^e ^ (e.g., size of the images, 
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image quality and download time. In addition to serving as an input interface, the slider bar (1140 11 50 1160 of Figure 
11) also acts as an output interface, displaying the feedback 390 that is received from the dynamic policy module 370. 
[0051] Figure 4 shows a block diagram 400 illustrating an example function of a multi-resolution cache 410 in a 
transcoding HTTP proxy system 400 in accordance with the present inventbn. Caches are useful in HTTP proxies to 

5 provide reduced response time for repeated data requests (by the same or different clients) for the same data object. 
A variety of methods may be used to assure that the cached data is up-to-date. In the example caching transcoding 
system 400, the multi -resolution cache 41 0 stores an original version of the data, as well as other possible forms of 
the data, including reduced resolution versions that have been transcoded for specific device types. 
[0052] As an example, we again consider the case of an image data object, but it is understood by those skilled in 

10 the art that these methods can be applied to other data types. We describe a method for storing, tagging and retrieving 
various forms of the data in the context of a caching transcoding proxy system. 

[0053] Referring to Figure 4, it is assumed that a JPEG image is received from the server 210 in response to a 
request by a client 230. This image is decoded from the lossy JPEG encoding format to a bit-map representation of 
the image. Those skilled in the art of image processing recognize that the JPEG encoding standard describes images 

15 using coefficients that weight Discrete Cosine Transforms (DCTs) so a compute-intensive decoding step is generally 
required to obtain the actual colors of each pixel. This bit-map requires a larger storage size than the original JPEG 
image. However, given an adequate size cache it may be worthwhile to store (via data path 420) this expanded size 
image so as to avoid the compute-intensive step of JPEG decoding when a transcoded version of the same image is 
later required but with different transcoding parameters than the first request. In a second step, based on the transcoding 

20 parameters (i.e., color depth, scaling factor, and the JPEG quality factor) the JPEG image is re-encoded. The image 
may be re-encoded as a JPEG image or in an alternative encoding format. The final transcoded version of the image 
is also stored in the multi -resolution cache 410 via data path 420. 

[0054] In the example embodiment, when additional data requests occur, the HTTP proxy 220 first checks its cache 
410 to see if an "up-to-date" version of the data object is available at the requested resolution or transcoding level. 
25 Each object in the multi-resolution cache 410 is stored with a version specifier that includes: a URL description, a time 
stamp for the data object, and the object characteristics. For a JPEG image the object characteristics include color- 
depth, scaling factor, and JPEG quality factor. An alternative embodiment has an indication that the JPEG image has 
been decoded and stored in its bit-map form, or that it has been converted and stored as a GIF with various charac- 
teristics. 

30 [0055] If an up-to-date version of the object is available with the requested type and resolution, then that version is 
returned to the client. If this is not available, but an up-to-date version of the object exists in either its original JPEG 
form or the decoded bit-map form, then this version is returned to the transcoder 240 with an indication of its charac- 
teristics. This enables the transcoder 240 to produce the desired version of the object, which is returned to the client 
230 and stored in the multi-resolution cache 410 via data path 420. 

35 [0056] It is noted that there are several extensions to this scheme that are obvious to those skilled in the art. One 
extensions uses an already transcoded version of the object, rather than to the original data object, to generate a 
further resolution-reduced version of the object. Methods for managing the different resolution versions of data objects 
in a cache are further described in R. O. LaMaire and J. T Robinson, "Conserving Storage Space by Means of Low 
Resolution Objects", docket Y0997308, US patent application filed February 13, 1998. 

40 [0057] Figure 5 shows an example flow diagram for the dynamic policy module 370 of Figures 3 and 4. Figure 5 also 
shows how the policy module 370 interfaces with the HTTP proxy engine 220 and the object transcoder 240 of Figure 
3. The methods described below apply to many content types including text, images, audio, and video. However, the 
following discussbn focuses on image data types only. It is evident to those skilled in the art that concepts and dynamic 
policies are applicable to other media types. 

45 [0058] Figure 5 shows that based on the response header received from the server 21 0, the HTTP proxy engine 220 
first determines the size of the object 510. If the content-type of the response is "image/*" 520, the proxy engine 220 
passes the handle for the object to the dynamic policy module 370 for further analysis. Inside the policy module 370, 
the size of the input object is compared against a pre configured threshold called "size_threshold" 530. If the objects 
is smaller than "size_threshold", or the content type is not "image/*" 520, then the object is not transcoded, but instead 

50 is forwarded 515 to the client without any content modification. Small objects (such as bullets, thumbnails, logos, etc.) 
found on the web are typically G|F objects which are already well compressed due to GIF encoding. Transcoding such 
objects does not generally yield further compression. 

[0059] We convert GIF images to GIF or JPEG images that are reduced in size and/or color-depth (the choice of 
GIF or JPEG as the end format depends on the image characteristics). In addition, we convert JPEG images to JPEG 
55 images that are reduced in JPEG quality, size, and/or color-depth. JPEG quality refers to a transcoding parameter that 
is used to determine the degree to which the coefficients of the Discrete Cosine Transformations used in the JPEG 
encoding standard are quantized. It has been found that the JPEG quality parameter is also a good predictor of per- 
ceived image quality. This parameter varies in the range 1 to 100, where 100 represents very high quality. JPEG images 
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fnund on the web tvoicallv have a JPEG quality parameter of 75. 

ESSi £ tarqe^ges the type of the image coding (JPEG or GIF) and the efficiency of coding are importam 
S in tSnsSinTaSion making. Since JPEG is a lossy compression method, size reduction is always poss,b e 
S^^S^aOot Similar quality reduction, however, cannot be applied to GIF files s-nce GIF .s a lossless 
^SSS^S^bO^b quality reduction, a GIF file must first be decoded and then Encoded as a q^- 

for coding maps, logos, and drawings while JPEG is more efficient for coding natural .mages. Converting GIF to JPEG 
is useful only when the original GIF image is not efficiently coded. ,u„ „«„ „f th» 

ncS? we define bits per pixel (bpp) as a measure of the compression efficiency. Bpp is computed as he ra 10 of the 
file^iz^e^TrM in piU. In processing step 540, X and Y dimensions and the bpp value of the .nput 

maoe are computed S parsing the image header. If the input object is of type -image/jpg" 550, transcoding .s always 
S^^S^^cJLnt typ'e is -image/gif 525, only those objects which yield a bpp ^ratic . larger han 
■aif threshold' 535 are transcoded. GIF files that are not very efficiently encoded yield a bpp value that 's larger than 
■2 "m eshold- -mus the decision step 535 is very effective in identifying compressible GIF files wrthhigh accuracy 

^Z^^S^- « >• to those ski,,ed in the art ihat other transcc f' n9 policies - as 9 

file truncation (for progressively encoded data) can be used for well -compressed GIFs. 

£>£ T/^mporLJ Aspect of the proposed invention is that decision steps 510-565 are earned out as soon as the 

tog h£££ receded' If the decision is to not transcode, image segments can be forwarded as soon as they are 
Reived from the server without incurring store and forward delay. Similarly, when transcoding is to be Performed 

Imag^ 

E5d fmage shtZe Transcoded. The po.icy function 565 is responsible for collecting input from three d, erent 
sources (image size predictor 375, bandwidth estimator 380, user preference selector 260) and subsequently selecting 
transit To parameters in accordance with the steps shown in Figure 8. The chosen parameters determine .the extent 
T^Zo^ress^ performed by the object transcoder 240. For example, the P^^^^ 
how much an image is downsamp.ed. Quantisation parameters control how an image is quanted ^°^<™" 
and/or the frequency domain. The number of colors in a color mapped .mage can be reduced, or a 24-b it color image 
may - be ^converted to 8-brt grayscale, or even a monochrome representation. The process of transcoding ,s performed 
in steD 570 and the output of the transcoder is forwarded to the client 230. 

£oM important aspect of the policy f unctbn 565 is the analytical framework for making transcoding decisions. 
T^nalyTbaT^ramework takes into consideration factors such as available bandwidth, type and size of the image 
Ler preferences^ 

objective of minfmizing response time for the user, but it is understood by those skilled in the art that using the same 
ISZSZZSLr optimization criteria can also be applied. We describe a method "iTttoal 
to transcode, and to what extent transcoding should be applied. The embod.ment described here.n ,s referred to as 

dynamic adaptation of transcoding parameters. rt u io ~t of <si 7 ^ 

So65] Figure 6 shows an example web request-response cycle and the response time of fete h.ngan object of size 
S^hVough a store-and-forward transcoding proxy 190. We define a store-and-forward .mage transcoder as an .mage 
transcoder which must wait to accumulate an entire input image before transcoding can beg.n on this .mage and then 
musTwait to qLne^te a transcoded image in its entirety before it is made available to be output. As shown .n Figure 
rthe'ig^S 620 of size S (bytej 670 is downloaded into ^ t ^«^^^^2^ 
connection with effective bandwidth Bsp (bits/sec). The transcoder introduces a delay DJS) ' to 
output image 630 of size SJS) 680. Both the transcoding delay 650 and output image s byte size 680 are denoted to 
be C3nt upon the input fmage's byte size S 670. The transcoded image is then transmuted over a P roxy«..ent 

connection having effective bandwidth 8^ , or i,irtinn achieved bv 

[0066] The policy function needs to weigh the cost (delay) of transcoding agamt any s.ze reduction 
transcoding For transcoding to provide benefits, delay introduced due to transcoding must be offset by the reduction 

^fmisiontime due to 'compression. For very low bandwkfth proxy-dient access JST 
time due to aaaressive image compression typically far outweighs the addition to response time caused by compute 

nTensTe U^Z ^er, Figure 7 shows that as the bandwidth of the proxy^.ient link increases, there comes 
TS!iS^SX^ 710) * ^ich it is no longer beneficial to transcode since the reduction ,n response .me 
du^to ires^rcompression decreases as a function of the bottleneck link's bandwidth, while the transcoding time 

SSST S^pTose R 0 - the response time of fetching a web object of size Sfrom the web server «*«™«^ 
unSoff. Snarly, let fl„denote the response time of fetchingthe transcodec Iversion 

the transcoding proxy For the purpose of the following discussion we assume that caching is not supported at the proxy. 
OOesT Cdien^erceived response time with transcoding turned off is the sum of the following three terms: 
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R 0 = 2 * RJT pc + 2 * R7T sp + S/minfB^, B sp ) 

[0069] RTTpc is the network roundtrip time latency between the client and the proxy and, similarly, RTT sp is the 
5 latency between the proxy and the server. Fetching the web object requires a TCP SYN/ACK exchange as well as an 
HTTP request/response, thereby contributing 2 * RTT^ + 2* RTT sp to the delay term. In addition, a web image incurs 
a transmission delay equal to the spread in time between arrival of its first and the last bits. Let min (B^ B^) denote 
the bottleneck bandwidth between the client and the server. In the absence of a proxy, the first and the last bits of an 
image will be spread in time by S/min (B^, B^). This spread corresponds to the effective transmission time of the 
10 image over the concatenated server-to-proxy-to-client connection. 

[0070] when transcoding is turned on, the proxy operates in a store and forward mode. 2 * RTT^ + 2* RTT sp is 
again the fixed component of the response time. D p (S) is the additional term that represents the transcoding delay. 
The resulting response time for the transcoded object can be expressed as: 

R p = 2 * RTT^+2* RTT^ + D p (S) + S©^ + SpfSyB^ 

Transcoding will reduce response time if R p < R Q . That is, 

20 D p (S) + Sm^ + S p rsyS pc < S/minfB^ B^) 

Clearly, when B pc > B Sf> R p is always greater than R & On the other hand, when B^ < B^, transcoding is useful if and 
only if: 

25 

D p (S) + sm sp < {S - s^sn/B^ 

[0071] The above equation precisely characterizes the regime in which transcoding reduces response time. Figure 
30 8 shows a flow diagram of an example policy function constructed using the analytical framework described above. 
810 marks the entry point of the policy function. The policy function BOO is called from 565 with the original image object 
size as one of the inputs to the policy function, variable S is set equal to the input object size and the quality factor q 
is initialized to the best possible initial image quality (for example, the quality of the input image). In step 830, the policy 
function 800 issues a query to the bandwidth estimator 380 asking for the estimated download time of the object size 
35 s from the specified server to the proxy, referred to as TrfS). It also asks for the estimate of the download time for the 
same object from the proxy to the client, referred to as T^S). Based on the logs of previous connections to the chosen 
destination, the bandwidth estimator returns an estimate of TrfS) and TgS). In the next step 840, the policy function 
queries the image size & delay predictor 375 to find an estimate of the transcoded image size S p (S). It then computes 
the estimated download time savings by subtracting the download time estimate of the transcoded image size T 2 (S p 
40 (S)) from T 2 (S) 850. Finally, in step 870 the two quantities (transcoding delay TrfS) - the download savings) and the 
target response time reduction are compared. If the first term is less than the second term, computation is stopped 
and the chosen quality factor q is returned as an output 880. Otherwise, the loop 840-870 is reentered with a reduced 
q value. 

[0072] It is noted that more efficient search techniques or variants of objective functions can be designed by those 
45 skilled in the art without departing from the spirit of the policy function framework presented in this invention. One 
extension would be to rearrange the terms in the policy equation as follows: 

Response time reduction(q) = {S - SpfSJJ/B^ - D p (S) - S/B^ 

so 

[0073] In the above equation, S p (S) is also a function of the quality factor (the smaller the output size, the poorer the 
quality). Note that the transcoding delay has been found to be effectively independent of the quality factor. There are 
two independent variables in the above equation: q, the quality factor, and the target response time reduction. Several 
different policies can be developed within the framework of the above equation. For example: 

55 

Minimize response time for all users; 

Maximize quality for a user specified response time constraint; 
Optimize overall system performance, not just one user. 
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delegating the responsibility of parameter selection to the ^™^ f ^ p0 nee to the end user. If users were to 
optima, transcoding parameters ^^ l^^eXT^ o^Zu^ optimal performance level, 
select these parameters nt ^|^ step 870 requires two addrtional 

[007S] Let us now consider the transcod.ng delay and | ,z | e ™°" ranscodi delay . DJS), that is incurred dur.ng 

Inputs: an estimate of the size of the ^^^X£ Z ^^ CM * ^ *™ ? 
the process of transcoding. An accurate predct io ' * ^ s d , ^ ^ loa d on the proxy. As was the 

the "mage content, type of coding used, ^^^^^Zh^ds to estimate these two quantities, 
case for the bandwidth estimators, the present rnmfton ^^^^ module in accor dance wfth the present 
[0076] Figure 9 is a block diagram of an example or the ^SS^^, of SJq) that are based on a large 
nventlon. As shown in 9! 0, we initialize our system ^255^5S2*d image s*e distribution functions 
benchmark set of images. The ^JS^SlT^npto points 920 to the distribute function 

behavior to dfflerent data sets that a client may be a ^ es ^ t ^ to estimate the de iay and output size and 

[0077] Thus, in the method described m 930, we "^^^S^J jrnages ^ he nce of forming a better 
use the bpp ratio as the crfteria for pertorrmngi ^tetSS weTJ GIF files, 
estimate o, the resuming transcoded s- ^^J^SSS^Z^ Ration and on-iine u P - 
[0078] Using similar statistical methods, D p (S) can be ' ^• m « stat i Stical work that the transcoding delay is well 

3-5 is responsible for providing these time TOt, ^ tes / he ^^ 

receive event at the proxy and performs f «f ca ' a ^ mending function can be 

of the network. Those skilled in the art of nt f^^^^^^ m feyer between the proxy application and 
implemented in several different ways One "^^^^^ any packe t filtering software. Genera.ly 
the" socket .aye, Another ^P'-^^ of bandwidth and down- 

the implementation ^<^^^^^^ZtL bandwidth montor 380 and the policy decision step 

e^lS 

module consists of two components, trace monrtor t 01 0 and ^^^J mM (or another stance of H) ® also 

:°rmi=^^ 



mation is recorded 



time when a new connection is established destination port number of each network connection, 



event, 

time when connections are closed. 



5 



I00 81] The statistica. ana*zer 102 0 maintains a database of a,, past ^^"^^T, 
Connection it aiso keeps a history of *m r^^'SSSI^^d^ time of future connections, 
on this history the statistical analyzer 1020 » able to make p time we use different heuristics. 

For predicting server to proxy download t.me and proxy to cl,en * °3^°_ rea TCP conn ections. It has been ob- 
S»2, Server to proxy traces exhibit ^^^^^^S^ estimation concepts such as 

function of this distribution, is returned as the download lame . j estimate, bandwidth constrained link, 

[0083] in contrast, proxy-to-clien. TCP behav '°" 8 ^^^ of a., act.e TCP connections 

which is typically the last hop 160-164. Because there is a bottleneck linK, tne gg a 
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to a client typically saturate the bottleneck link. Thus, for providing proxy to client download time estimates, bandwidth 
estimator aggregates all active connections to a client into a single group. For each group of connections it plots a time 
vs number of bytes plot and then performs a linear curve-fit on the data. This technique provides accurate estimate of 
the current available bandwidth between the proxy and the client Although this value changes with time, the oscillations 
s in most cases are bounded. For example, bandwidth monitor can easily detect whether the client is connected by a 
10Mb ethernet, 14.4 modem, or 28.8 modem by analyzing at the output of the curve fitting algorithm. The predicted 
values are not exact (20- 30% deviation from the correct values), but those skilled in the art recognize that accuracy 
of bandwidth estimation can be improved by refining the estimation algorithm. 

[0084] Consideration is now given to the transcoding user interface. Transcoding user interface is the interface 
io through which the user can dynamically change the tradeoff between image quality and download time. It is represented 
as the item 260 in Figures 2-5. Figure 11 shows an example implementation of the user preference interface in ac- 
cordance with the current invention. This user interface is represented well as a linear slide bar, which can be imple- 
mented as a Java applet, to allow users to continuously vary the image quality. As shown in Figure 11 an option for 
selecting color (vs. gray scale) is also provided. Further, an option for selecting autopilot can be used in the following 
is two ways: 1 ) as an indication to the proxy that it can automatically decide whether or not to perform transcoding, based 
on the current estimates of the client-proxy and proxy-server link bandwidths and currently available CPU resources 
as discussed above, or 2) as an indication to the proxy to try and maintain the current download times that the user is 
currently experiencing (i.e., by varying image quality at the proxy to compensate for dynamic bandwidth changes that 
are detected by the bandwidth estimation subsystem), or both 1) and 2) . On occasions when the user requires a full- 
20 fidelity version of the data, they can easily request the refined data by adjusting the slide bar (downwards) and request- 
ing the higher quality data. Typically, we find that almost all of our web browsing can be done by transcoding images 
to a sixth or tenth of their original size, and only rarely do we actually find a object, such as a map, that we need to 
request a refinement of. Note that alternative HTML-only interfaces can have been invented for supporting non-Java 
devices such as windows CE based platforms. 
25 [0085] Now we consider streaming image transcoding. The above description is generally applicable to a store-and- 
forward transcoding proxy. Following, we describe conditions under which it is beneficial for a streamed image trans- 
coder to engage in transcoding. A streamed image transcoder is an image transcoder which starts writing out image 
data encoded in an output format before having fully read in the complete input stream of bytes corresponding to the 
entire image encoded in the input format. 
30 [0086] In Figure 12 we illustrate an example of the reasoning behind bur algorithm by using a timing diagram. The 
input image arrives as a stream of bits spaced apart by The streaming image transcoder will take a group of G 
bits for transcoding, incurring a small store-and-forward delay D 1 . The group of bits are then transcoded into a group 
of G p output bits : incurring a delay If D^<D 7 , then the image transcoder can convert each input group G to its 
corresponding output group G p before the next input group G needs to be processed. In this case, the streaming image 
35 transcoder's internal memory requirement is bounded. However, if D^>D 1t then the image transcoder will not be able 
to process input bits fast enough. In this latter case, given a continuous input stream, then the image transcoder's 
internal memory requirement grows without bound, i.e. the image transcoder's finite-length internal RAM buffers will 
overflow. Therefore, we desire than the transcoding delay D 2 satisfy D 2 <D 1 . Clearly, D 1 = G/B^. To find let D p (S) 
= the predicted image transcoding time for an image of S bits (D p (S) actually depends on other parameters, such as 
image content and dimension, but we use D^S; for simplicity of notation). Then, D 2 ~ D p (s)/(S/G). To avoid overflowing 
the RAM buffer, the group transcoding delay must satisfy D p (S)/(S/G) < G/B Sfy or 



40 



45 



D p (S) < S/B ps (Condition A) 



[0087] Assuming that Condition A holds true, then the output transcoded groups G p will be uniformly spaced by a 
delay equal to D v The transmission channel can send each transcoded group of bits G p in time D 3 = Gp/B^ In case 
i), we illustrate D 3 (i)<D v i.e. each output group Gp, can be sent before the next output group is ready for transmission. 
In case ii), D^ii) > D p so that the output transmission link cannot send the produced bits fast enough to keep the output 
50 queue empty. In case ii), the transmission link's output queue grows without bound given a continuous stream of trans- 
coded bits, causing overflow for finite-length link buffers. Therefore, we desire that the delay caused by the transcoded 
output group size Dg(i) < D v Clearly, D 3 (i) - Gp/B^ To avoid overflowing the transmission link's output buffer, the 
transcoded output image group size G p must satisfy Gp/B pc < G/B sp) or 
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c > B p /B (Condition B) 
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wherec = grou P „press^ 

t N whe"e N is a number less than 1 Normally, the compress.on ratio is be djsadvantageous . 

always be satisfied, m this case, on* Condition A providing an upper bound on 

in fact, when the proxy-server link is T<B^ Expansion of an image may occasionally 
the ratio of expansion allowed ^ ^ranscoded .mage, n^ 

be necessary when o TtSSSZ. and when format conversion will not cause 

when such format convers.on w,ll increase the chances o I ouner o ^ g ^ ^ ^ grQUp Gp 

buffer overflow. For example, if Bsp = ' Ops> "pc - * °p* ^I'-neck i e 6 >B M then Condition B says that 

tor transcoding and/or allow less aggressive C °™P™*™. hQd for a transco ding proxy to facilitate browsing 

[0 089] It is thus an aspect of the present ^^^^^^ The method includes receiving an HTTP 
between client devices and servers '°" n ^*^^ the GET request for the object 

GET request from a client devices for an object stored at one o the ^servers^ t a ^ ^ ^ d&/ . ce 

to the s'eiver, receiving the object from the "™*Z^^ choosing a set of transcoding 

examining contents of the object, »nnq llZT^^nX^^ form to the c.ient, and/or examining 
parameters, forming a transcoded om ,°V"! t ^roandwS oeleen the server and the proxy as well as between 
network characteristics includ>ng estimating netwo *^ d ™™^ ) and as well as de lay between the proxy 

tne proxy and the client, and/or - timat '"^ decoding performed on the object, 

and the client device, and/or providing a f oedbaok to the user^ bo determining dimensions of the object, 

and/or the step of examining inc.uding determining the * dimensions of the image object 

and/or calculating the compression * determined by the bpp ratio of the 

are determined by area of the image in square puce and ™ "*L P and strea ming transcoding, thus allowing form.ng 
image object The present invention .allows be* ' i ^^^^^ ca i UlB . This method can be applied to 

before the step of forming a transcoded '^V^. 

the methods of the present invention. .... .. e inupntion je ma de for particular arrangements of steps, the 

[0092] It is noted that although the descr.pt.on of the arrangements. For example, the in- 

Lent and concept of the present ^ nt '°" 

dynamic and dynamic implementations. 



50 Claims 



55 



, ^thodforatransccrtin^^^^ 

connected va a communication network, the method comprising. 

receding a request from one of the c.ient devices for an object stored at one of the servers, 
forwarding the request for the object to said one of the servers, 
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receiving the object from said one of the servers, 

examining preferences specified by a user of said one of the client devices, 
s examining contents of the object, 

examining communication network characteristics, 
choosing a set of transcoding parameters, 

10 

forming a transcoded form of the object, and 

sending the transcoded form to said one of the clients. 

A method as recited in claim 1 , wherein the network characteristics includes bandwidth, and examining network 
characteristics includes estimating network bandwidth between said one of the servers and the proxy as well as 
between the proxy and said one of the clients. 

A method as recited in claim 1 or claim 2, wherein the network characteristics includes delay, and examining 
network characteristics includes estimating delay between said one of the servers and the proxy as well as delay 
between the proxy and said one of the clients. 

A method as recited in any preceding claim, further comprising providing feedback to the user about a level of 
transcoding performed on the object to form the transcoded form. 

A method as recited in any preceding claim, wherein the step of forming a transcoded form employs dynamic 
adaptation. 

A method as recited in any preceding claim, wherein the step of forming a transcoded form is started before the 
step of receiving the object from said one of the servers is complete. 

A method as recited in any preceding claim, wherein the step of sending out the transcoded form is started after 
processing an initial fraction of the received object, and before the step of receiving the image object from said 
one of the servers is complete. 

A method as recited in any preceding claim 1 , wherein the step of sending out the transcoded form is started before 
the step of forming a transcoded form of the object is complete. 

A transcoding proxy system for facilitating browsing between a plurality of clients and a plurality of servers con- 
nected via a communication network, the proxy comprising: 

an HTTP proxy engine to receive a request from one of the clients for an object stored at one of the servers, 
and to fetch the object from said one of the servers, 

an object transcoder to form a transcoded form of the object using a set of parameters for transcoding, 

a dynamic policy module to determine the set of parameters of transcoding, 

an image size and delay predictor module to gather characteristics of the object, 
so 

a user preference module to gather quality preferences specified by a user of said one of the clients, and 

a bandwidth estimation module to estimate available network bandwidth, 

55 wherein said dynamic policy module dynamically adjusts the parameters of transcoding using the input re- 

ceived from the image size and delay predictor module, user preferences module, and bandwidth estimation 
module for the purpose of improving satisfaction for the user, and the transcoding system provides feedback 
to the user about the level of transcoding performed. 
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10. A computer program comprising computer program instructions to cause a computer to perform the steps of the 
method as claimed in any of claims 1 to 8. 
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(57) The present invention provides methods, de- 
vices and systems for dynamically adjusting transcod- 
ing parameters so as to increase the benefits of trans- 
coding. Methods of adaptation are designed to cope 
with the variability of network characteristics and of the 
size of transcoded images. The invention also provides 
a method and apparatus to enable the transcoding 
proxy to adjust a quality-size tradeoff on a per-image 



and/or a per-client basis. The adaptive transcoder 
chooses different parameters for each object, and pro- 
vides performance improvements. The invention further 
provides a general framework for making policy deci- 
sions taking into account available bandwidth, content 
and type of image, and user preferences. The invention 
also includes methods for generating feedback about 
the choice of optimal transcoding parameters to the us- 
er. 
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